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ABSTRACT 



This paper presents initial findings from a 2-year study to 
identify the core knowledge and skills necessary to teach in a 
standards -based system. The study was part of the Standards-Based Teacher 
Education Project (STEP) , which focused on the development of frameworks and 
materials for preservice and practicing teachers. High school teachers and 
higher education faculty formed "co-development teams" in each of six 
disciplines committed to implementation of standards -based instruction and 
assessment in one or more of their classes. In collaborative work groups, 
they shared pedagogical practices, broadened content knowledge, and learned 
new ways to engage students in learning. Culminating statewide institutes in 
the spring of 1997 and 1998 provided the opportunity for co- development teams 
to cross-score classroom performance assessments, determine levels of student 
proficiency, and verify one another's judgments of proficiency. This paper 
summarizes initial findings from participants' reflective inquiry. The study 
found that the voices of STEP teachers engaged in the actual implementation 
of standards provided clear imperatives for those who formulate educational 
policy and design teacher preparation and professional development programs. 
The overarching imperative is that teachers are the primary agents of school 
reform and should be included in all stages of design, implementation, and 
improvement of the standards-based system. (Contains 47 references.) (SM) 
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Teaching in a Standards-based System: How Teachers' Voices Can Influence 
Policy, Preparation and Professional Development 

INTRODUCTION AND CONTEXT 

This paper presents some of the initial findings from a two-year study to identify the core 
knowledge and skills necessary to teach in a standards-based system. This study was part of the 
Standards-based Teacher Education Project (STEP) which focused on the development of 
frameworks and materials for pre-service and practicing teachers. High school teachers and 
higher education faculty formed “co-development teams” in each of six disciplines committed to 
the implementation of standards-based instruction and assessment in one or more of their classes. 
In collaborative work groups, they shared pedagogical practices, broadened their content 
knowledge, and learned new ways to engage students in learning. Culminating statewide 
institutes in the spring of 1997 and 1998 provided the opportunity for co-development teams to 
cross-score classroom performance assessments, determine levels of student proficiency and 
verify one another’s judgements of proficiency. We summarize initial findings from participants' 
reflective inquiry and suggest implications for policy makers, teacher educators, professional 
developers, and educational researchers who are involved in standards-based reform. 

Oregon’s Reform Movement: Standards from a K-16 Perspective 

Oregon’s reform movement is driven by a policy framework that links the 
implementation of standards and assessments with teacher pre-service preparation and 
continuing professional development. The state also has a unique K-16 perspective, as the only 
state in the nation with board adopted standards aligning student performance in elementary, 
middle, and high school with college admissions. 

Four major initiatives frame this integrated vision of a seamless standards system bound 
to teacher preparation and professional development. First, the Oregon Educational Act for the 
21st Century, passed in 1991 (amended in 1995) mandated the design and implementation of a 
comprehensive system of content and performance standards and assessments benchmarked at 
grades 3, 5, 8 and 10. Students earn Certificates of Initial Mastery (CIM at grade 10) and 
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Advanced Mastery (CAM at grade 12+) by demonstrating proficiency on state tests and 
classroom performance assessments that are aligned with the standards. Second, in response to 
this legislative initiative, the Oregon State Board of Higher Education adopted a policy in 1994 
to develop a proficiency-based admission standards system for entry into the state's seven public 
universities linked with these K-12 standards and assessments. The Joint Boards, comprised of 
the Oregon Board of Education (pre-school, K-12, community colleges) and the Oregon State 
Board of Higher Education (public universities), formally adopted this alignment and a timeline 
for implementation in March 1998. 

The remaining two framing initiatives recognize teachers’ critical role in policy as agents 
of implementation of standards-based reform. In 1997, the Oregon State Legislature 
unanimously enacted Senate Bill 124 with the support of the Teachers Standards and Practices 
Commission (TSPC — the state licensing agency), the Joint Boards of Education and educational 
leaders across the state. This new design addresses the preparation of teachers for Oregon's 
standards-based schools and includes three-year initial and five-year continuing teaching 
licenses. The practicing teacher has a significant role in designing and documenting a Continuing 
Professional Development Plan (CPD) that addresses six domains of professional competence. 
The CPD along with successful teaching provides evidence to initiate and renew teaching 
licenses. Finally, Oregon’s Senate Bill 880 further links student learning to teacher performance 
by replacing guarantees of tenure with two-year contracts and renewals based on proven ability 
in standards-based teaching (TSPC, 1999). 

Assessments from a K-16 Perspective: The Role of Teacher Judgement 

While the policy framework provides the broad brushstrokes for a standards-based 
system, the aligned K-16 standards and resulting assessment system create a complex mandate 
for classroom teachers as agents of implementation. At the classroom level, the culture of 
standards and performance assessments challenges traditional teacher practice and moves the 
once private act of judging student performance into the public arena as multiple choice tests 
provide only part of the accountability picture. 

The Oregon State Assessment System for K-12 consists of three parts: (a) state multiple 
choice tests of content knowledge (e.g. life science, physical science), (b) classroom work 
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samples (e.g. scientific inquiry), and (c) on-demand performance tasks graded by other teachers 
at the state level (e.g. writing to a prompt: My most memorable experience...). In four content 
areas, scoring guides assist teachers in determining student proficiency in core cognitive 
processes that are the “gateway” for student acquisition of knowledge in these disciplines: 
writing (English), problem solving (mathematics), scientific inquiry (science) and social science 
analysis (social science). These scoring guides provide the framework for teacher judgement of 
student performance on classroom work samples and on-demand performance tasks. The Oregon 
Department of Education (ODE) assumes responsibility for structuring multiple choice and on- 
demand performance assessments, and for monitoring student performance in these four 
disciplines while leaving the determination of proficiency in second languages and visual and 
performing arts to the discretion of local districts. Linked to this K-12 assessment system is 
admission into higher education via the Proficiency-based Admissions Standards System 
(PASS). 

PASS requires that students demonstrate proficiency in all six content areas through a 
combination of multiple choice tests (national and state) and “collections of evidence.” These 
collections include classroom tasks, performances (e.g. oral interviews in second languages), 
teacher-made tests, and long term projects (e.g. research paper, scientific investigation). The 
Oregon University System (OUS) assumes responsibility for structuring the framework of 
proficiencies and accompanying criteria for proficient performance. However, classroom 
teachers, as agents of implementation, are responsible for teaching to the standards and working 
with students to generate collections of evidence that indicate sufficiency (the range of work 
addresses the criteria for meeting the standard) and proficiency (the performance on that work is 
at the level necessary for college entry coursework). PASS collections in English, mathematics, 
science, and social science for college admission generally contain work samples that are 
assessed with state scoring guides and that are also used to meet high school requirements for the 
Certificates of Initial and Advanced Mastery. In order to study the comparability of teacher 
judgement, the PASS staff convenes “Verification Institutes,” where teachers cross score 
collections of work, determine the level of student proficiency and compare their judgments with 
those of their colleagues. Unlike most states where K-12 teachers are engaged in similar cross 
scoring activities, Oregon’s emerging system of standards and assessments relies heavily on 
teacher judgement of student proficiency across the K- 1 6 system . 
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Providing a Forum for the Voices of Implementation - Classroom Teachers 

Beginning in 1994, the directors of PASS relied on high school teachers as well as higher 
education faculty to assist with the design of standards and assessments for admission into the 
state’s seven public institutions. Successive iterations of PASS standards and criteria have 
resulted from numerous collegial reviews and Verification Institutes attended by community 
college and university faculty and approximately 300 teachers representing over half the high 
schools in Oregon. While the design of PASS required extensive teacher and faculty input, the 
actual implementation of this complex policy initiative linked to K-12 reform, required changes 
in classroom practice. The Standards-based Teacher Education Project (STEP) was undertaken to 
determine the knowledge and skills necessary to teach in a standards-based system. Over a two- 
year period, 44 high school teachers and university faculty participated in a reflective study of 
their professional practice. The intent was that the actual experience of practitioners engaged in 
implementation could inform the further development of teacher preparation programs and 
materials, continuing professional development and policies related to standards implementation. 

Using practitioner experience as a referent for policy implementation is both pragmatic 
and practical given that teachers are the agents of implementation held to high levels of 
accountability for successful student performance in ambitious systemic reform efforts such as 
Oregon’s. Policymakers may fail to acknowledge teachers as initiators of change who are able to 
define problems and devise solutions (Tyack and Cuban, 1995; Schulman and Sykes, 1983). The 
exclusion of such voices is both perplexing and problematic. When they view teachers simply as 
a “conduit for instructional policy, not as an actor’’ policymakers tend to “invest a great deal 
more in the creation of control systems for teaching than they do in the development of teacher 
knowledge’’ (Darling-Hammond, 1990, p. 345). By contrast, enlisting the support and skills of 
teachers as key actors in reform might be seen as a positive kind of tinkering, adapting 
knowledgeably to local needs and circumstances, preserving what is valuable and correcting 
what is not (Tyack and Cuban, 1995). 

A one-dimensional “top-down’’ approach to policy implementation invites failure by 
disengagement (McLaughlin, 1987). As one STEP social science teacher from a high school 
aptly stated: “Never underestimate the ability of teachers to stonewall or sabotage change 
imposed. We can all adopt a new terminology, series of reporting forms, and then go back to our 
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classrooms, close our doors, and do exactly what we’ve always done.” Even though teachers 
may have full discretion over decision making within their classroom walls, they are often 
excluded from management decisions beyond these walls (Shedd and Bacharach, 1991). In 
essence, they may be relegated to the status of “lower participants” in the formal organization of 
their school, whose voices are “difficult to assess” by those who are in positions of authority 
(LeCompte, 1993, p.lO). Or, as Gitlin and Meyers (1993) suggest: 

Teachers’ stories are rarely given a forum in the public domain. This is not particularly 
surprising given that teachers often are not expected to formulate policy or influence the 
direction of educational practice but rather are expected to take the wisdom of others and 
find ways to put these theories into practice, (pp. 51-52) 

In stark contrast to this culture of disengagement, STEP provided a forum for the voices 
of teachers engaged in the implementation of standards (Bodone, 1997). Teachers, it was 
reasoned, could through careful study of their own practice, clearly articulate the knowledge and 
skills needed for teaching in a standards-based classroom. The “legitimate knowledge” (Gitlin et 
al., 1992) generated through this reflective and collaborative work could then best inform the 
design of teacher preparation programs, professional development and policies guiding teacher 
education and accountability. 

/ 

THEORETICAL FRAMEWORKS 

The concern with voice has come to have special relevance for the place teachers occupy 
and the role they play in school restructuring and reform and in how research knowledge 
about teachers and their work is generated. What part do teachers play? What say do they 
have in educational change? How well or poorly are their perspectives represented in the 
discourse of policy and research on education? These are issues to which the concept of 
voice speaks especially well. (Hargreaves, 1996, p. 12) 

The words of Andy Hargreaves are particularly relevant to our work with high school teachers 
and higher education faculty engaged in the implementation of standards and assessments within 
their own classrooms. The voices of individuals actively engaged in such reform efforts 
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illuminate the possibilities and challenges inherent during implementation (Bodohe, 1997). We 
approached our study from the belief expressed by McLaughlin (1987) that 

organizations don't innovate or implement change, individuals do. Individuals responsible 
for carrying out a policy act not only from institutional incentives, but also from 
professional and personal motivation. Change is ultimately a problem of the smallest unit. 
(p.174) 

With that in mind, we consider the general concept of voice, teachers’ voices, their place 
in qualitative and educational research, and the possibilities as well as limitations. 

The Concept of Voice 

The concept of voice has become significant over the last two decades in philosophical 
and theoretical discourses such as postmodernism (Giroux, 1992; hooks, 1992; McLaughlin and 
Tierney, 1993), feminism (Middleton, 1993; Patai, 1988), and critical pedagogy (Freire, 1970; 
Kanpol, 1994). Following these lines of reasoning, the individual (the self) is the engine of the 
discussion, the principal informant of what is going on. Individual voices are spoken out, each 
holding their own truth and value (Bodone, 1997). In those interpretive thinking processes the 
event is eventless unless voices talk about it and attach a special significance to it (Middleton, 
1993). 

Beyond the testimonies and stories that voices can tell there is the breaking of a long 
silence which, historically, has affected groups of people in our societies whose lives and 
thoughts about those same lives, have been given little or no attention. Coming out of this silence 
is the first step towards recognition from others. What is very important is that voices are heard, 
as emphasized by bell hooks (1989): 

The challenge for teachers, as well as for other silenced groups, is not to emerge from 
silence to speech, but to change the nature and direction of our speech. To make a speech 
that compels listeners, one that is heard, (as cited in Gitlin, 1994, p. 192) 

Once voices are heard, they cannot be ignored. They exist and are part of the discourse. 

Or are they really? Qualitative research has voice in the center of its design and with it numerous 
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ethical and philosophical discussions among qualitative researchers about validity, authenticity, 
whose voice it is, ownership of the text, etc. Too often, the voice(s) of informants (or researched) 
is taken for granted; we (researchers) hear it, but do we really listen to it? And if we do, what do 
we do with it? Do we interpret it or do we give it a chance to be expressed in its entirety, for its 
own worth and sake? Or do we simply replace it? (Bodone, 1997) To this particular point, bell 
hooks (1990) speaks powerfully: 

I am waiting for them to stop talking about the other... it is not just important what we 
speak about, but how and why [italics added] we speak. Often, this speech about the 
other erases the other: I can talk about you better than you can speak about yourself. No 
need to hear your voice. Only tell me about your pain. I want to know your story. And 
then I will tell it back to you in a new way. Tell it back in such a way that it has become 
mine. Re-writing you. (pp. 151-152) 

In the context of education, do we give teachers' voices a fair chance to be heard and 
empowered to actually impact the changes that affect their schools? Or, do we simply, as 
researchers and/or policy makers collect those voices to illustrate theories about schooling and 
give a "practitioner" feel and make up to the general field of educational research and reform? 

Teachers' Voice(s) 

One premise that frames our discussion is a belief that teachers are involuntarily silent 
and voluntarily silenced when it comes to changing education (Bodone, 1997), i.e. an oppressed 
group. The Brazilian educator Paulo Freire (1970) described the oppressed as those who are not 
able or allowed to "say his or her own words, to name the world". Teachers fit this description. 
Their world is that of education, but their words do not name it or define it. Their role and 
expertise are often limited to the classroom with no real opportunities to expand beyond this 
setting. 

Teachers' voices have not been absent from the literature. There have been many books 
written about teachers' stories and experiences (Ashton-Wamer, 1963; Kozol, 1967) or collection 
of teachers' texts (Ayers, 1995; McLaughlin and Tierney, 1993). Teachers have much to say and 
will say it. What is often missing is a recognition of their stories, a validation that those 
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experiences and teachers' interpretations of those experiences can inform and influence policies 
that are made to change schools or teacher education programs. Robyn Russell, a teacher who 
collaborated with Gitlin (1994) on an alternative methodology to research teachers' experiences, 
had this comment: 

The struggle between silence and voice has been lifelong for me... Society told me to be 
seen and not heard, like some naughty child, while an inside whisper begged for a 
listening ear. It was within this state of fluctuation that I began a journey into further 
study of the educational system. This story is about my chosen project of developing 
teacher voice; but more than this, it is about my own travels in this previously uncharted 
terrain: to speak and to be heard, (p. 1 89) 

For teachers to develop their voices means that the content of their words is knowledge 
worth considering, a legitimate knowledge (Gitlin and Meyers, 1993) that has been acquired 
through years of professional experiences. Back in the early 1980s, when scientific and 
quantitative data held the answers of all questions, what teachers knew and could say about 
schools and their profession carried little weight, as confirmed by Darling-Hammond (1990): 

A decade ago, when my colleague Arthur Wise and I embarked on a study of teachers' 
views of educational policies, we were exploring largely uncharted territory. At that time, 
the idea of talking to teachers as a way of illuminating the effects of policy was viewed 
by many as irrelevant, at best, and methodically unsound, at worst (After all, teachers 
could hardly be presumed to be "objective”; neither were they viewed by many as an 
important part of the policy process), (p. 340) 

This devaluation of teachers' stories has stigmatized many who, disillusioned and 
deprived of what they thought was their legitimate knowledge, don't wish to develop their 
voices beyond their classroom doors. Instead, “they have grown to anticipate the continuance 
of school structures and mandates, instigated without their input and often in the face of their 
objections” (Gitlin, 1994, p. 192). Or as Lortie (1975), in his book Schoolteacher portrays it: 

Schooling is long on prescription, short on description. Although books and articles 
instructing teachers on how they should behave are legion, empirical studies of teaching 
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work - and the outlook of those who staff the schools - remain rare, (as cited in 

Goodson, 1992, p. 3) 

Qualitative research via its various methods — focus groups, interviews, observations — 
goes inside classrooms, talks to people who are there daily (teachers and students), hears them, 
observes occurrences and phenomena that makeup the realities of schools and brings people 
together to reflect upon those occurrences and phenomena. It grasps the nuances and the 
emotional dimension of human experiences (Bodone, 1997). In educational research qualitative 
methodology has helped to move from "taking teachers and students for granted and treating 
them as classroom furniture, to seeing them as thinking and possibly disputatious human beings" 
(Ravitch, as cited in Bacharach, 1990, p. 13). 

Possibilities and Limitations 

STEP focused primarily on the collection of teachers' thoughts and learnings about the 
implementation of standard-based reform in their classrooms and schools, and solicited their 
suggestions for improving teacher education programs. Teachers' contextualized and revealed 
experiences became a vehicle by which practice could inform theory (Bodone, 1997). Policy can 
then respond to the "real" problems faced by those who practice education everyday. This is 
easier said than done, because in reality educational theory and practice exist in different 
realities. 

At the Ninth Education Trust National Conference (November 1998), policymakers and 
educational researchers met to discuss barriers encountered while linking research to policy in 
education and found that: 

- policy aims at the "possible", whereas research aims at "perfection"; 

- scope: policy deals with large spaces, research with microcosms; 

- policy is interested in what works, not why, which is the main question of qualitative research 
(Bodone, 1998, personal notes). 

This argumentative session revealed a clear lack of communication and understanding 
between these two worlds which both look at the same reality, i.e. that of education. At times, it 
felt like the two parties spoke two different languages, two different English languages, revealing 
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even semantic differences in the terminology they used to describe issues (Bodone, 1998, 
personal notes). 

Qualitative researchers, who value the “thinking” possibilities of teachers and enable 
their voices, are becoming hermeneuts — interpreters. They render the voices of the unheard in a 
language accessible both to teachers themselves and to a wider and presumably more powerful 
audience (LeCompte, 1993) i.e. policy makers and other researchers. Teachers' voices are 
moving to the forefront of educational research for there is a need in the social sciences, and 
particularly in education, for interpretive texts... [because] "it's time for theory building" 
(Catapano, 1991, p. 14). 

What teachers know determines what they do, which in turn, determines what a student 
learns. And the learning outcomes determine what needs to be changed or improved. It is a 
circular, dialectical relationship between the teacher and student, and no one better than them 
(teacher or student) knows what goes on in that educational process. If expert knowledge is what 
policy makers and educational researchers are after, then it seems logical to elicit teachers' (and 
students') voices through their stories to get the "authentic" information that can best explain the 
mechanisms at work as well as the personal and emotional underpinnings. Carter (1993), a 
teacher educator, confirms this: 

For many of us, stories capture, more than scores or mathematical formulae ever can, the 
richness and indeterminacy of our experiences as teachers and the complexity of our 
understandings of what teaching is and how others can be prepared to engage in this 
profession, (p. 5) 

The recommendations for change, through teachers’ narratives, come from the trenches 
where transformation is processed from the ground up (Lieberman, 1995), a recognized criterion 
for successful reform. Lynde Paule, the evaluation coordinator for PASS, noted that to achieve 
coherence between educational theories and educational practices, the prescriptive words of 
teachers, as practitioners of the theories, are vital (informal conversation. May 1997). Teachers 
are more likely to listen to and implement prescribed practices if what they need to do is voiced 
by one or many of their own. The prescription is experiential and furthermore, originates from a 
shared professional environment and culture. Hong (1996) who examined one year of the effects 
of reform in one school via a qualitative design, explains the following: 
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There is a huge emotional gap between an academic exposition of the difficulties of 
school reform and a tension-charged meeting of parents and teachers to discuss a 
proposal for multiage grouping. A polite seminar discussion does not begin to touch the 
bewildering pain a teacher may have when confronted with instructional changes that 
upend 20 years of practice, (p. xv) 

It is a natural human phenomenon, to trust the words and actions of one who shares 
similar working conditions, frustrations, joys, apprehensions, and daily complexities. Scientists 
trust other scientists, doctors other doctors. They read each other's studies and share expertise in 
their journals. This does not mean that outside influences or recommendations cannot or should 
not filter into the world of teachers. To the contrary, we believe it is a valuable exchange, as long 
as it is an exchange, a dialogue. Local circumstances and contexts influence the global, larger 
political and research picture, and vice versa. As teachers' voices inform policy and research, in 
return research studies and policy materials could be regularly disseminated so that teachers are 
aware of what goes on outside their classrooms, in other schools, in other states, and around the 
world. 

METHODOLOGY 

The STEP research has relied on qualitative methods while drawing on principles of 
action research and educative research to elicit teachers’ voices and document the experiences of 
these co-development team participants in the process of implementing standards-based 
practices. We utilized several data collection methods such as journal entries, individual 
interviews, focus groups, team meeting proceedings, email questions and documents review to 
identify the knowledge, skills, approaches and strategies used by teachers in the process of 
transitioning from a traditional teacher centered classroom to a learning environment which is 
designed around and focused on student learning. 
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Research “With” not “On” 

Action research is about dialogue, about teachers and educators working and thinking 
together to change their practice for the better. Here, contrary to a more empirical tradition, 
theory does not determine practice. Teachers are not required to fit their practice into the 
theoretical molds of some outside educational researchers mandated by even outsider 
policymakers (Bodone, 1997). To the contrary, teachers are encouraged to make their own 
decisions on what is most appropriate for their practice and educational philosophy: research is 
done WITH them, not ON them (McNiff, 1988). 

Teachers who participate in an action research project become aware of their teaching 
skills, philosophy, and of the daily experiences that they share with students, parents, and 
colleagues. They reflect upon these, critique them, and act upon them, opening themselves up for 
changes, for innovations, with the goal of an improved practice. McNiff (1988) describes the 
process in these words: 

The method itself of action research is elegant. It involves a self-reflective spiral of 
planning, acting, observing, reflecting and re-plarming. It requires teachers to be acutely 
aware of a sense of process, and to refine their perceptions to account for that process. 

Far from being ad hoc, action research raises to a conscious level much of what is already 
being done by good teachers on an intuitive level. It enables teachers to identify and 
come to grips with their practice in a humane way, which is at once supportive and 
critical. As a headmaster said of his staff: 'It liberates teachers from their prejudices and 
allows their instincts to blossom.' (p. 7) 

Teacher Learning via Educative Research 

Teachers base their "legitimate" knowledge on experience (Gitlin et al., 1992) having had 
little opportunity to engage in other forms of inquiry, in part for lack of time and resources. 

While theoretically-based knowledge has been given recognition and legitimacy, teachers and 
their experiential expertise are often placed last at the bottom of the hierarchical order of what 
counts and what does not in educational policy and research. Gitlin et al. believe that one way to 
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encourage the validation of more than one legitimate knowledge lies in how we research, in the 
kinds of methods we use to collect and validate people's knowledge. They propose educative 
research 

that values both experiential knowledge and knowledge produced through systematic 
inquiry and therefore attempts to broaden notions of expertise.... And what difference if 
any would this sort of method make? For example, would teachers' voices be heard, and, 
if so, would these voices encourage school change and challenge the narrow and 
oppressive features of our school system? (p. 2) 

Educative research aims at developing teachers' voices with their active participation in 
this process of documenting their insights. Teachers become responsible for the very research 
that is being done on their practice, giving them ownership of what to ask and how to ask for it 
as well as to make them accountable for the understanding of and transformation of their own 
practice. Gitlin et al. (1992) explain that the term educative refers to a process that brings 
individuals together in such a fashion that participants are involved in determining research 
questions and expected to benefit from the overall "educative" experience (p. 7). There is the 
dialectical relationship between their thinking and acting on their outside and inside 
circumstances: teachers are able to question the outside meaning makers of their professions, 
such as educational research and theories, policy makers, and the various control systems put in 
place to monitor their actions. Meanwhile, they are also looking inwards, inside their own 
thinking and practice, to generate the needed knowledge and critical reflection that will allow for 
the transformation of their daily experiences and empower them as creators and masters of their 
art, teaching. 

In both educative and action research, teachers voices are allowed and able to develop, to 
move beyond the mere act of speaking into (critical) reflection and action, into praxis (Freire, 
1970). They are not generic and do not become "the" voice of teachers; each of them, consonant 
or dissonant, is part of this legitimate knowledge that the group has generated while working and 
thinking together. When teachers are viewed as researchers, they regain their status of "experts" 
whose knowledge can be seen as a legitimate source of data for informing policies and practices 
that impact their work (Bodone, 1997). 
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Participants Organized into Co-development Teams 

The data were collected over a two-year period, from the fall of 1996 to the summer of 
1998. There were 44 secondary teachers and higher education faculty participating in this 
reflective study of professional practice. These voluntary participants were from a purposeful 
sample (Patton, 1990) representing 17 high schools and five higher education institutions. These 
schools were geographically representative of many areas in the state of Oregon. The project 
teachers were selected and invited to participate as they were already members of the larger 
PASS project and therefore experienced with standards-based reform. 

The teachers and faculty were organized into co-development teams of three to eight 
members representing six disciplines including: English, mathematics, science, social science, 
second languages, visual and performing arts. Each co-development team was comprised of 
beginning (2-5 years) and veteran (up to 30 years) teachers, working in rural and urban settings, 
and serving diverse student populations and communities. The teacher and faculty participant 
groups reflected diverse ethnic backgrounds, personal characteristics, and professional 
experiences. This enabled us to access a multitude of descriptive "teachers' voices" rather than a 
more singular, prescriptive "teacher's voice" (Hargreaves, 1996). 

These high school and higher education co-development teams were a unique feature of 
STEP, bringing together into collaborative work groups two sectors of education that rarely 
interface. The teams provided opportunities for focused conversations on standards-based 
teaching and learning, the sharing of instruction and assessment materials and both the informal 
and formal evaluation of student work. Participants were treated as teams of experts, each of 
whom was engaged in the implementation of a standards-based K-16 system and contributed to 
the wisdom gleaned from practice as a teacher - beginning, veteran, high school, and higher 
education. 

The Focus for Inquiry: 

What knowledge and skills are needed for teaching in a standards-based system? 

The participants' reflective study began within the context of an action research cycle 
focused on the knowledge and skills necessary to teach in a standards-based system. Initially, the 
inquiry ranged broadly around the question of what knowledge and skills are needed for teaching 
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in a standards-based system? The members of each co-development team were free to select any 
one of a number of PASS proficiency standards in their content areas (math, English, science, 
social science, visual and performing arts and second languages) as a focus for instruction and 
assessment. Each participant captured their experiences in a journal using hand-written or 
electronic recording and collected classroom artifacts (assignments, assessments, student work) 
to illustrate standards-based practices. Meeting four to five times a year as co-development teams 
and once as an entire STEP project group, participants’ learnings emerged in these general 
thematic areas; Teacher knowledge and skills, instructional strategies, student learning and shifts 
in thinking. At the end of the first year, focus groups conducted with each co-development team 
examined specific practices such as how teachers determine course content, select instructional 
strategies and evaluate student progress. General reflections included what impact standards have 
on teaching behavior and students’ learning. 

During the second year, participants elected to adopt a common framework for their 
inquiry that was organized around a general set of teacher practices in a standards-based system. 
These practices included: Targeting specific standards for teaching and learning, planning 
instruction and assessment around that target, teaching with a variety of strategies to ensure 
student progress toward that standard, assessing student learning across time with multiple 
measures, verifying their judgements of student proficiency with other team members through 
the cross-scoring of student work and reflecting upon their experiences. Using co-development 
team questions (Table 1) derived from the first year's data, participants were more focused, and 
as a result delved further into their own pedagogical practices, specifically describing that which 
seemed to enhance student learning. Each co-development team also selected a specific PASS 
proficiency standard as a common focus for the team’s inquiry and members committed to full 
implementation of the common framework of teacher practices and these standards in one or 
more classes. The intent was that each team would be speaking from a similar frame of reference 
about the actual experience of standards-based teaching. 
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Table 1 : STEP Co-Development Team Questions for Year 2 



Target 


• What do you think (the standard) expects students to know and be able to do? 

• What types of information will you use to determine what students need to know and 
be able to do? 


Plan 


• How will you plan to assess students’ knowledge and skills? 

• How will you plan to teach (the standard)? 

• How will you help students make meaning of (the standard)? 

• What are the knowledge and skills students will need to study the standard? 

• How will you assess students’ prior knowledge and skills related to the standard? 

• What specific content knowledge and skills will you need to teach to and assess (the 
standard)? 


Teach 


• What instructional methods/strategies/resources will you use to teach (the standard) in 
your classroom? 

• How will you connect (the standard) to what students know and where they need to be . 
to meet the standard? 

• How will you approach the varying levels of knowledge and skills among your 
students? 


Assess 


• How will you know if a student is “proficient” or not? 

• What evidence will you use to determine this? 


Verify 


• How does your judgment of student proficiency compare with other teachers’ 
judgments of that student’s work? 

• How will you know that you share a common understanding of student proficiency in 
(the standard)? 

• How will you work with colleagues to develop a common understanding of (the 
standard), review and cross-score student work, and verify judgments about student 
proficiency? 


Reflect 


• How do you use evidence of student performance to improve your professional 
practice? 

• How will you modify, refine or renew practices in designing curriculum, instruction 
and assessment? 

• How will you further your own content knowledge and pedagogy? 

• What role will colleagues play in furthering your professional development? 



The proficiency standard(s) selected by each co-development team addressed: (a) writing 
and literary analysis (English), (b) problem-solving in a sub-discipline such as algebra 
(mathematics), (c) scientific inquiry in a sub-discipline such as biology (science), (d) 
performance within a social/historical context in the arts such as music (visual and performing 
arts), (e) social science analysis in a sub-discipline such as history (social science) and (f) oral 
proficiency and written expression (second languages). In addition to sharing teacher materials 
and student work at team meetings, participants also assembled collections of student work 
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Data Collection 

Multiple methods of data collection were used to capture the learning generated by each 
co-development team’s experiences as standards implementation revolved around an action 
research cycle. The sources of data collected include the following items: 

1) Journal entries - these included general reflections on standards-based teaching within the 
context of class(es) where teachers agreed to focus instruction on a specific standard, assess 
student performance toward that standard, and generate student work that would then be 
shared with other teachers in the group along with classroom tasks, scoring guides and other 
materials. Journals were shared during four of the group meetings that occurred between 
September and June of the first year. 

2) Focus group responses - conducted in each of the six discipline groups in June of the first 
year, questions specifically focused on recommendations for teacher education programs 
based on participants' reflections of the previous year over their teaching in a standards-based 
classroom. 

3) Responses to co-development team questions - derived from the journal process of the first 
year, these 20 questions in six categories (target, plan teach, assess, verify, reflect) focused 
conversations around a cycle of classroom activities. This cycle led to a "Verification 
Institute" where all participating teachers met in their content area groups with student work 
and verified or confirmed one another's judgements on student's proficiency (or lack thereof). 
Specific questions were assigned via email, summarized for subsequent group meetings, then 
served as a point of departure for further discussion. Group meetings occurred every other 
month from September through June of the second year. 
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4) Co-development team proceedings - individual co-development team meetings occurred 
regularly and consisted of four to five meetings per year at various locations. The meetings 
convened for two to five hour sessions and all proceedings were audio taped and eventually 
transcribed. During year one, participants and facilitators had open-ended meeting 
discussions addressing all the aspects of their standards-based implementation experience. 
Colleagues brought teacher and student materials (i.e. scoring guides, tasks, curriculum 
plans, student-parent communication, etc.) to share, peer critique and co-develop. The 

■ second year the proceedings were guided by a more structured agenda including the 
revisiting of email question responses, focus questions for discussion, materials sharing, and 
discussion encompassing their questions and issues as they work with standards. 

5) Teacher materials - during the first and second years, participants in each group shared 
classroom tasks, activities, scoring guides, and samples of student work. From the discussion 
of these materials, they derived strategies and techniques for improving their practice in a 
standards-based classroom. 

6) Teacher surveys - the STEP high school teachers were a subset of the approximately 100 
PASS teachers from 18 Oregon high schools who completed surveys in the spring of 1997, 
and again in 1998. The instrument examined teacher knowledge and levels of use of 
standards. The analysis of responses also included possible differences between STEP/non- 
STEP teachers as a group and by content area. Higher education faculty participating in 
STEP did not complete the surveys. 

7) Student surveys - these surveys were administered in the same PASS sites and at the same 
time as were the teacher surveys, and analyzed in a similar fashion. 

Two research assistants working with the project director systematically collected data, 
using the various methods outlined above for all six of the co-development teams in each content 
area (math, English, science, social science, visual and performing arts and second languages). 
The two assistants also coordinated and facilitated the co-development team meetings, each 
supporting three groups. All meetings and face-to-face interviews were audio taped and 
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transcribed. The hand- written journal entries were also transcribed and the email responses from 
participants were prepared for coding. 

Studying multiple participants and institutional settings and employing multiple methods 
of data collection over time, allowed for triangulation of the data, and in doing so greatly 
strengthened the study’s usefulness for other settings (Marshall and Rossman, 1995). The 
importance of multiple sources of evidence in order to provide and maintain a reliable chain of 
evidence about the topic in question was highlighted by Yin (1994). He suggests that these 
multiple sources essentially provide “multiple measures of the same phenomena” (p. 92). 

The major purpose of the open-ended focus group and individual interviews was to "learn 
to see the world from the eyes of the person being interviewed" (Ely, Anzul, Friedman, Gamer 
and Steinmetz, 1991). Seidman (1991) wrote: "Interviewing provides access to the context of 
people's behavior and thereby provides a way for researchers to understand the meaning of that 
behavior . . . [It] allows us to put behavior in context and provide access to understanding their 
action" (p. 4). The focus group interviews and meeting proceedings encouraged discussion and 
expression of various points of view, therefore providing participants with an opportunity to hear 
opinions and understandings and then form their own (Marshall and Rossman, 1995). The 
teacher materials shared during team meetings corroborated information collected through 
journals, meetings, focus group interviews and individual interviews, and email question 
responses (Yin, 1994). 



Data Analysis 

The project director and the two research assistants systematically and collaboratively 
analyzed the data which were collected from all the various sources across the six co- 
development teams. Previously, all the raw data were formatted and prepared for entry into a 
database which was organized for the analysis software. Qualitative Solutions and Research 
Non-numerical Unstructured Data Indexing Searching and Theorizing (QSRNud*ist). This 
software package aided us with our analysis by supporting the processes of coding data in an 
index system, searching text, and searching patterns for coding and theorizing about the data 
(Gahan and Hannibal, 1998). 
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Next, a system of content analysis that involved identifying, coding, and categorizing the 
main themes in the data was adopted. The QSR Nud*ist program helped manage the raw and 
coded data so that a preliminary categorizing of participant responses was accomplished. The 
program enabled us to code the data based on these categories and after coding it, to examine 
these categories for patterns across the six content teams and within the individual groups. As 
we analyzed the data from year one, we looked for “reoccurring regularities in the data” (Patton, 
1990) which produced patterns that could then be sorted into categories. We then coded 
participants’ response patterns across the six content areas into a total of 24 emergent categories 
framed with the following titles; student learning, teacher behaviors, instructional strategies, and 
shifts in thinking. We then worked back and forth between the data and the classification schema 
to verify the meaningfulness and accuracy of the categories and data placement (Patton, 1990). 

We identified emergent categories from the year one coding to frame our five new 
categories. Following the collection of year two data, we proceeded to code and analyze the data 
by applying the same strategies as previously employed. We coded data into the categories of 
target, plan, teach, assess, verify and reflect. The last step in the analysis for this paper was to 
focus on the verification category in order to identify major themes. Participants’ quotes were 
carefully selected to illustrate each of the three major themed findings described in the next 
section. 



FINDINGS 



The 60 co-development team meeting transcriptions, six focus group interviews, 
individual interviews, three statewide all-participant retreats, journal entries, 250 electronic 
responses to focus questions and various teacher materials comprised a substantial body of data. 
For the purposes of this paper, the findings reported are those that were coded during analysis 
into the verification category of the framework (described by six categories) for standards-based 
teaching that served as the focus of year two activities. 

Teachers’ voices on verification are of particular interest given the central role teacher 
judgement plays in the Oregon assessment system for both K-12 and college admission. As 
discussed previously, the Oregon University System through PASS bases judgement of student 
readiness for college entry level on assessment information generated through a combination of 
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performance assessment (collections of student work in specific proficiency areas) and multiple 
choice tests (national and state). High school teachers are responsible for teaching to the 
standards and working with students to generate collections that indicate sufficiency (the range 
of work addresses the criteria for meeting the standard) and proficiency (the performance on that 

work is at the level necessary for college entry coursework). Teachers who are participating in 

\ 

the development of PASS may verify their judgements of collections of student work with other 
teachers in their schools and with other teachers from around the state at PASS Verification 
Institutes or, in the case of STEP teachers, with co-development team members. For the purpose 
of college admissions, teacher judgements of student proficiency will be recorded on a 
supplemental transcript report in each of the six content areas on a 1-5 scale with 3 indicating 
“meeting proficiency.” Judgements of 4 or 5 must be verified by other PASS teachers and higher 
education faculty. 

Within STEP, the verification of teacher judgement of student proficiency was both a 
formative and summative process. During co-development team meetings, participants focused 
their conversations on the PASS standards in their content area. They described how they were 
designing teaching and learning activities around the target standard and how they were working 
with students to generate a variety of “evidence” that when collected together, provided a clear 
picture of proficient performance. As trust was developed among team members, teachers shared 
classroom tasks, scoring guides and samples of student working requesting specific feedback on 
their classroom materials, planned activities, strategies and approaches and whether or not it 
appeared that students were approaching proficiency. The PASS Verification Institutes, held in 
May of each year, provided a summative experience where STEP teachers’ collections of student 
work, along with those of over 125 other teachers, were judged on the merits of sufficiency of 
evidence and proficiency of performance. 

As PASS is phased in over the next six years to full implementation, the consistency of 
judgement in what will be a high stakes assessment system is of central concern. Moving beyond 
the notion of simple consistency of judgement, Gipps (1994) calls for “comparability, that is, 
equivalence in the application of standards across assessment tasks, assessment occasions, 
students, classes, schools and school years” (p. 28). Furthermore, this degree of consistency 
requires “shared experience and understandings within a community of judgement,” (Wolf, 




Tell, Bodone, and Addle (1999). Teaching in a Standards-based System: How Teachers' Voices Can infiuence 
Poiicy, Preparation and Professionai Deveiopment 



23 



22 



1995, p. 71). In other words, this common understanding must be embedded in the minds and 
professional practice of teachers. 

Three major themes emerged from the study data, describing in some detail the various 
frameworks and supports that must be present for teachers to achieve common understanding 
and reach comparable Judgement of student proficiency: 

• a clearly articulated framework of standards and assessments supports teachers’ common 
understanding of student learning and comparable Judgement of student proficiency; 

• training and professional development opportunities that are collegial in nature enhance 
teachers’ content knowledge, pedagogy, and capacity to reach common understanding; and 

• leadership is the critical factor in embedding standards-based teacher practices into the 
organizational structure of schools. 

These themes are discussed in greater detail below. 

1). A Clearly Articulated Framework of Standards and Assessments Supports Teachers’ 
Common Understanding of Student Learning and 
Comparable Judgement of Student Proficiency 

1.1. Criteria that are linked to standards provide a means for teachers to reach common 
understanding of student progress toward the standards. 

We found that across the content areas of English, math, science, social science, visual 
and performing arts and second languages, teachers generally spoke to the need for having an 
accompanying set of criteria that guides teachers’ judgement toward an understanding of whether 
or not the student has demonstrated proficiency. These criteria promote a common vision of 
student performance and ensure that teacher Judgements are based on this common vision rather 
than on idiosyncratic preferences. The challenge for teachers is that these criteria do not always 
exist. In English, science and math where PASS criteria are currently developed in the greatest 
detail, high school (HS) teachers and higher education (HE) faculty attest to the link between 
specific guidelines and consistent Judgement with these comments: 

If we can come to common ground first before even looking at a student work, and if the 
scoring guide is clear enough to be used in Justifying one's assessment, then it is possible 
to come to an agreement. This process will need to be maintained. We tend to adapt our 
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expectations to what we see; therefore, when teachers assess their own work without any 
opportunity for 'refocusing', then they may be straying from what is generally agreed 
upon as proficient performance. When the classroom teacher and an outside evaluator 
(from another school) can agree, then the student is clearly proficient. (English, HS 
Teacher) 

When I score essays, I have to admit that I look for what the essay does well. Several of 
the other readers look for an accumulation of problems, and after a certain (unspecified) 
number, score the essay as 'not proficient'. I have done 'mini-reads' with one or two 
faculty this year, as well, and we often agree on scores. This comes from discussing in a 
more focused way the criteria for scoring and the expectations we have for good or 
exemplary performance. There are certain faculty, though, who do not agree with me on 
standards and criteria for grading. They seem to want to give credit for 'effort' which 
drives me crazy. (English, HE Professor) 

When I attend a verification institute, I am amazed how much correlation of judgment 
there is between my judgments and other teachers. I know that there is a common 
understanding of the standards and that teachers understand the need to be consistently 
critical of student work in regards to standards. (Science, HS Teacher) 

I don't really talk about student proficiency to other faculty. But in PASS Verification 
Institutes, we mostly do agree as we are given specific guidelines on what is proficiency. 
(Mathematics, HE Professor) 

In the area of Second Languages, PASS criteria are designed around oral and written 
communication linked to a national fi'amework of proficiency standards rather than designed 
around proficiency in literature translation. Our STEP teachers share this orientation toward 
proficiency in communication. Over a two-year period, the process of assessing student 
performance against specific criteria served to establish a larger vision of student performance 
and how that might be linked to classroom instruction and assessment. A second language 
teacher and a faculty member respond with the following thoughts: 
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I cannot say for certain that my judgment of student work will match other people's 
judgments all of the time, but I found it reassuring that when we had enough evidence to 
make a judgment, that we could almost always agree whether the student was proficient 
or not. To what degree the student was or not (exemplary) seemed to be a mute point in 
most respects, as long as we agree on the main issue of proficiency. For now. I'm pleased 
if another professional in my field and I can agree that a certain student has crossed the 
threshold into proficiency. (Second Language, HS Teacher) 

When we gather across the languages, those of us who work with the same standards and 
assessment tools, the comparison becomes much less problematic. We can work in 
considerable detail to describe a possible new curriculum, and we can share teaching 
techniques and activities accurately up and down the proficiency scale, because we have 
common measurements and goals. (Second Language, HE Professor) 

In social science and the arts, PASS criteria are less clearly defined. These content areas 
are essentially collections of unique sub-disciplines (e.g. history, economics or drama and choral 
music) united in the PASS framework only by a common process for demonstration of student 
proficiency (e.g. social science analysis and performance of the arts). This causes teachers to rely 
to a greater degree solely on their own experience or knowledge of their sub-discipline to make 
judgements about student proficiency. This notion is reflected below: 

I thought on the social studies at the PASS Verification Institute that the scoring guide 
was not specific enough to really make a judgment. So, you just take this general feeling 
that you have from the paper... And we all kind of said, Yeah! This kid can make it to 
college or not, this kid is quite not there! But in terms of saying why, it's hard! (Social 
Science, HS Teacher) 

It is difficult to come to an agreement if the group assessing operates from a different 
level of expectation or standard of excellence. (Arts, HS Teacher) 

When I work with people other than choral directors to assess student work which uses 
choral/vocal music as the medium, the range of assessments begins to broaden. Likewise, 
when I attempt to assess student works which use other arts strands, my assessments 
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deviate further from the norm as established by the instructors who specialize in that 
medium. The fact that there are common elements which are basic to the arts does not 
diminish the fact that there are tremendous issues of technique for the use of those 
co mm on elements which are specific to each art form and often take a lifetime of study to 
acquire. (Arts, HS Teacher) 

Teachers may look at the same piece and have difficulty judging that with the same eyes 
even though we have the same language in front of us. Even if “balance”, the term 
“balance” is understood to another teacher and it's understood by me, when we look at 
the same performance, we have the same criteria in front of us, we are still going to judge 
that differently. (Arts, HS Teacher) 

1.2. Scoring guides and exemplars of proficient performance provide tools forjudging 
student work and building common understanding. 

Across all six content areas, we found STEP teachers embraced the use of various types 
of scoring guides that called for various approaches for arriving at a judgement. A formal state- 
sanctioned instrument, like the ODE writing scoring guide, requires judgement on each of six 
analytical traits. Less formal guides include the PASS proficiency standard scoring guides that 
provide detailed criteria at the “meeting proficiency” level (see Appendix). PASS calls for a 
holistic judgement on whether or not these criteria are represented in the student work directly, 
or that sufficient evidence is presented for the rater to infer that the student would be proficient 
in all criteria. Informal teacher-created scoring guides were also shared over the two-year period, 
by members of the six co-development teams. Regardless of the type of scoring guide or 
judgement called for, STEP teachers acknowledged the value of these tools for enhancing 
common understanding and tempering individual variation in judgement (or particular biases) 
towards one’s own students as evidenced in these remarks: 

1 will know that 1 share a common understanding when we can look at work and ‘score’ it 
or grade it within one grade or number, using exactly the same scoring mechanism and 
making sure we understand the mechanism the same way. There is a universe of 
interpretation that comes into the grading or evaluating activity, though, and one of the 
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most important steps for me is to make sure everyone involved understands the scoring 
guide the same way. When I am confident that we all understand the scoring guide the 
same way, then the activity can proceed without too much arguing about the features 
we're looking for. (English, HE Professor) 

Scoring guides have allowed me to understand and implement a consistent evaluation 
process which in turn has given me a common understanding of student proficiency. 
(Social Science, HS Teacher) 

I most often agree with colleagues who have knowledge of the scoring guides; In other 
than PASS work. I'm considered a hard grader but offer many more than one chance for 
students to demonstrate their knowledge. (Mathematics, HS Teacher) 

I haven't scored any of my own proficiency work without being worried that I'm being 
too hard on the kids or too easy. As their teacher, I want them to do well, and I worry that 
the score I give them (when it's not cross-scored) is an inflated score. It is always on the 
back of my mind that if the work were to be cross-scored, the student would receive a 
different score altogether.... How can you be assured that you're scoring accurately? 
(Second Language, HS Teacher) 

Along with scoring guides, exemplars of student performance complete the “picture” of 
student proficiency, as confirmed by an English faculty participant: 

I think any "common understanding" of standard will have to be re-negotiated with every 
reading and assessment experience. The value of "range finders," or sample papers that 
resemble features found in the whole collection of samples is high in this kind of system. 
If we have the standard and continually refer to it, that is good. What tends to happen, 
however, is that we "drift" even within a single-sitting assessment experience. Heaven 
knows what happens over time and with many different age levels and types of work to 
assess. Thus, experiences where we get together to evaluate and discuss the same work, 
to agree on the salient features, and then to read and assess will be vital. 
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Use of scoring guides and exemplars were frequently related in teachers’ comments about 
the purpose of the judgement being made. For PASS, teachers are asked to judge the readiness of 
the student to do entry level college coursework. This level of readiness, in general, was 
particularly difficult to define in the arts where there is no consensus within or across the sub- 
disciplines as indicated by these arts high school teachers: 

Where should the bar be placed for every student who is applying for admission to 
university? If this determination is made within each art form, will there be an attempt to 
make the height of the bar comparable across the arts? 

What will be the absolute minimum that every student should know about the arts to 
survive and make a meaningful contribution to society in general? 

It has really bothered me and it really is a challenge to me to try to figure out, what do I 
believe would be appropriate for students who are not going as music majors? 

Lacking clear exemplars and defined performance levels, high school Arts teachers 
explained how they addressed this challenge when immersed in the verification process by 
shifting the focus to their vision of the archetypal entering student to reach common 
understanding of proficient performance: 

What happened at our table was that they felt several students that I had judged as 'not yet 
proficient' actually met proficiency as far as they were concerned and that my classroom 
standard was too high for PASS. My colleagues had to remind me that I should be setting 
proficiency levels for the generic student and not the third or fourth year theatre student 
whose collections I was submitting. 

I think we were much more realistic about our expectations this year. We used "Joe 
Football" as the guide to help us remember that although we are attempting to raise the 
bar, there are many students who are quite average (or even untalented) in the arts. We 
cannot expect every college bound student to hold the lead in a play and design costumes 
and know theatre history, all at a mastery level! This is a change in thinking from last 
year! 
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In the second languages, two distinct schools of thought offer opposing views on 
proficiency and the means by which to assess it. One university faculty member of a co- 
development team shared the challenge of using common tools for judgement within these 
differing constructs: 

Often it is impossible to compare, since they are using radically different concepts of 
language proficiency - at least when they are wearing their in-class "Grammar hats". 
When language teachers take off those hats and judge as normal human beings ("Can this 
person speak the language?"), we can often reach good agreement. 

While scoring guides and exemplars of student proficiency provided the tools for 
reaching conmion judgement, they are still the means to an end - the verification process. A 
second language teacher states his secondary school view: 

The first time I have participated in an actual cross-scoring of student work was at the 
PASS Verification Institute. This was perhaps the most exciting workshop I have 
attended to date. To me, it was significant and FINALLY we were sitting down with 
scoring guides in hand, however crude the scoring guides may be, and holding student 
work up to a common standard. I found this to be excellent and stimulating. I would 
have liked to spend an entire week doing this work. 

2). Training and Professional Development Opportunities that are Collegial in Nature 
Enhance Teachers’ Content Knowledge, Pedagogy 
and Capacity to Reach Common Understanding 

( 

2.1. Collegial activities provide the avenue for purposive professional development. 

As professional working relationships were developed across STEP co-development 
teams, a genuine appreciation for collegial activity emerged as well as a recognition that this type 
of activity rarely occurs in schools as they are currently structured. As one high school social 
science teacher lauded the end of “splendid isolation” in his department, other teachers who are 
isolated by virtue of their discipline, e.g. the sole drama teacher, welcomed the opportunity to 
join a larger statewide collegial group. These collegial activities are purposive, providing a 
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vehicle for enhancing teacher judgement, expanding frameworks for assessment and deepening 
common understanding of proficiency that will ultimately impact classroom practice. Several 
disciplines discussed their collegial work on standards as illustrated by these statements: 

Talking with peers is the way I think all of us will know when we are making appropriate 
judgements. This is new to many schools since I usually go for a whole day without 
seeing another teacher. This "talking with peers" thing is a revolutionary concept 
especially since there is only one of me in this district and very few others understand 
that theatre is more than fancy costumes and cool sets. We need to make the dialogue 
with the other Arts teachers meaningful; we need to speak a common language. (Arts, HS 
Teacher) 

I will know we share a common understanding when we all sit down with the same set of 
goals (that is, a genuine proficiency orientation), and proceed to hammer out the 
curricular system for our program, constantly proposing and judging with reference to 
language proficiency. (Second Language, HE Professor) 

With every proficiency except E, it would seem that our ability to know a common 
standard can only come through collegial work. Arts assessment is still in its infancy and 
although there are some terrific models out there, I think we need to hammer it out 
among ourselves as most of the present models are a response to national standards, and 
PASS proficiencies go beyond product outcomes into process skills. (Arts, HS Teacher) 

The experience with the STEP co-development team caused several individuals to 
envision how they might tackle the challenge of introducing collegial learning in the context of 
their own school or campus. It is interesting to note that there is always a ‘standards focus’ to 
this collegial activity, to the verification process, to reviewing student work and to improving 
comparability of judgement as the following participants stated: 



I will try to work with colleagues in my discipline to work up a common scoring guide so 
that students throughout the school know that teachers are consistent. I will work with 
teachers in different disciplines to reinforce the concept of standards and to reinforce the 
use of scoring guides, and the relevance and connectedness of various disciplines. I will 
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ask to score work from other resources in the discipline, and have other teachers score my 
student work to help verify judgments about student proficiency. (Social Science, HS 
Teacher) 

Faculty, too, even when they feel insecure about "judging" student writing or speaking, 
can gain confidence when they share their insights in a group. Respecting each other's 
ideas about student performance is one of the best ways to get faculty to share their 
students' work. One of the most important ways to get faculty to verify their judgments is 
by making them feel secure enough to share their students' work." Another way would be 
for faculty to sit in on each other's student conferences and listen to what features of 
writing or speaking or critical thinking get emphasized in these meetings. This might be a 
good "mentor" model for establishing consistency. (English, HE Professor) 

I hope that within the next year, I will be able to work on helping teachers to feel more 
comfortable. I see there is a need for that within buildings, districts and statewide so that 
all students are being presented to higher education institutions from the same 
fundamental background without inflated grades or meeting of the standard. (Science, HS 
Teacher) 

Student work can be introduced to my teachers. There needs to be a common decision to 
target a particular standard and everyone would bring student work to be scored. No one 
would know whose work belonged to whom, but we would score all of it together to see 
if there is consistency between teachers. (Science, HS Teacher) 

2.2. Teachers’ knowledge provides the foundation for comparable judgement. 

Across all content areas, the development of teachers’ content knowledge was frequently 
cited as a vital factor in reaching comparable judgements, as this high school arts participant 
explains: 

I have found judgments are comparable especially when the breadth and depth of the 
knowledge base is similar with those in collaboration. The quality of assessment and 
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effectiveness of scoring depends largely on the knowledge of the group comparing the 
student's work. 

In addition to content knowledge as we traditionally understand it, STEP teachers 
referred to a broader expertise marked by a deep understanding of teaching and learning in one’s 
discipline, defined by Schulman (1987) as “pedagogical content knowledge”. This notion was 
seen as a necessary attribute of teacher professionalism that contributed to comparable 
judgements. STEP teachers noted an interesting aberration in scoring. Teachers who lacked 
knowledge tended to rate student proficiency at a higher level. A few examples of this 
observation are included in the following statements: 

If I am with teachers who know mathematics, then we are pretty consistent. If I am with 
a group of teachers who have not kept up with mathematical education over the last 1 0 to 
20 years then they tend to have higher scores on papers. (Mathematics, HE Professor) 

I know that when my student teacher began scoring writing samples, he gave everyone 5 
and 6 when I was giving them 2 and 3. It took him a lot of practice and discussion. He 
admitted that it 'sounded good' or 'that the student really meant something else'. This is 
where the standards are so important: Could the student perform at a specified level? Did 
we set a predetermined level of achievement before scoring? Did the student have prior 
knowledge of how they would be assessed so that they could have opportunity to reach 
the minimum level of achievement? (Second Language, HS Teacher) 

Overall, validity is an important issue but a paradoxical one. The paradox lies in the 
observation that surely the most vulnerable aspect of proficiency-based assessment is 
validity, while in practice it is not a problem at all. I should qualify that. It is not a 
problem among higher education faculty and some of the better high school teachers, and 
even then it is only not a problem in an abstract context. What I worry about is that many 
teachers do not have the understanding (as opposed to knowledge) of content needed to 
have assimilated professional norms, and when faced with real students in a real life 
situation, they will allow their judgment to be clouded by a myriad of external factors. 
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just as it happens with grades in the current system. We MUST build in external 
validation and quality assurance to make this work! (Mathematics, HE Professor) 

3). Leadership is the Critical Factor in Embedding Standards-based Teacher Practices into 

the Organizational Structure of Schools 

3.1. Leadership controls the time and the opportunity for collegial learning . 

A recurring theme throughout the STEP project was that the lack of building level 
administrative support or knowledge of standards actually inhibited standards based activities of 
teachers. Our findings show a general recognition that the traditional structure of schools does 
not allow for the many collegial activities that were deemed necessary by STEP teachers in order 
to reach comparable judgements. These activities included collegial reviews of student work, 
classroom tasks and scoring guides as well as opportunities for conversations around teaching 
and learning in a standards-based classroom. Clearly, the structure of the school day and 
allocation of resources were two areas viewed as being out of the control of the classroom 
teacher and in the hands of administrators. The following responses give a general idea of these 
perceptions: 

The only way I will be able to work with colleagues (in a worthwhile and productive 
manner) is if I have the support of my department and administrative faculty. Having the 
entire staff in agreement working with PASS would be the icing on the cake. The entire 
school needs to be involved and hours provided (not beyond the working day) to 
collaborate and develop this common understanding, to review and cross-score and verify 
judgments. (Arts, HS Teacher) 

The standards provide for us the words, and our school administrators just need to give us 
the opportunity to have the conversation. I think this is the most successful way of 
making sure we are on the same page with regard to student verification. (Arts, HS 
Teacher) 

Even at the university, a lot will depend on the administration providing an opportunity 
for faculty conversations. I will try to create the structure to ensure successful 
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conversations in the Math department at , through the department assessment team 

and in university studies. (Mathematics, HE Professor) 

If possible, we need to have regular time scheduled for teachers to meet and discuss 
student work, score and cross-score, and collaborate about judgments of student work. 
One half day per month would be enough for teachers to use in preparation for using the 
standards and in student scoring. The only way for inexperienced teachers to begin to 
understand the standards is to use them in scoring student work. (Mathematics, HS 
Teacher) 

In our school, we have done some of this in writing proficiencies. Time has been set aside 
to do this. There is simply not enough time to cross grade many papers given the time 
constraints which are presently upon us. The fact is that standards based teaching in 
social studies requires much more time to begin with, and there already is precious little 
time to get all the work done. (Social Science, HS Teacher) 

There's not a tradition of sharing and we are all feeling very much time deprived, and so 
we Just, you don't have this time to long term strategic planning, you Just figure out, 
what's my next unit going to be. (Social Science, HS Teacher) 

3,2. Leadership must focus the content and process of professional development throughout 
the school year on teaching to standards and reaching comparable judgements on 
student work. 

STEP teachers tended to request professional development experiences that mirrored 
their experiences within the project. They clearly indicated that responsibility for allocation of 
resources to this effort falls to school and district administrators. Teachers also speculated that 
school administrators would benefit from participation in professional development activities 
with their staff Many STEP participants indicated that their co-development team and project 
experiences were the most significant and meaningful professional development they had 
encountered in their teaching careers. The voices below represent several viewpoints regarding 
professional development: 
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First of all, all teachers must have solid content and process training. They must be 
excellent readers, writers, and thinkers themselves. Not only do they have to model it; 
they have to know it well enough to be able to articulate its various facets. This 
absolutely must be supported, and, in fact, forced, upon us as a staff by our 
administrators, at both a building and at a district level. It must be part of our in-services. 
(English, HS Teacher) 

Teachers must have time at the beginning of the year to target a proficiency and decide 
how it will be done throughout the year. Teachers must have time as the year progresses 
to start looking together at student work, to see (a) how the students are doing and (b) 
how to verify judgments made by each individual teacher. Unless we have this kind of 
building and district-level support, it is going to be impossible to do the work we need to 
do to verify proficiency. It will be up to each teacher and they will bow to grade pressure 
the way they always have. If we do have time for in-services, we will need administrators 
who are just as interested as the teachers are to lead and facilitate the process. Everyone 
in the building needs to look at student work. Everyone needs to converse about it with 
others in the field. Administrators must become involved in order for the process to work, 
and all teachers must become involved also. We need to pull in people from the periphery 
so they can see what goes on. We have proven in verification and calibration sessions that 
people from all different fields can judge work proficient equally well, as long as they 
can have conversations about it. (English, HS Teacher) 

Our department has put in a request for time this summer, as well as hoping to work 
together as an action team (during our delayed starts) next year to tackle these issues. The 
key to all this seems to be time, energy, and motivation! (Science, HS Teacher) 

CONCLUSION 

The voices of STEP teachers engaged in the actual implementation of standards provide 
clear imperatives for those charged with the formulation of educational policy and the design of 
teacher preparation programs and professional development to support that implementation. We 
believe the overarching imperative is that teachers are the primary agents of school reform; 
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therefore, they should be included as co-developers in all stages of design, implementation and 
continuous improvement of the standards-based system. Other educational researchers also 
support this finding (Darling-Hammond, 1990, 1996; Hong, 1996; Shedd and Bacharach, 1991) 
and policy makers who are attempting to implement successful reforms in schools nationally and 
internationally. The International Commission on Education for the 21st Century (1996), in its 
report to UNESCO, noted the following: 

No reform can succeed without the co-operation and active participation of teachers. 
Attempts to imposed educational reforms from the top down, or from outside, have 
obviously failed. The countries where the process has been relatively successful are those 
that obtained a determined commitment from local communities, parents, and teachers, 
backed up by continuing dialogue and various forms of outside financial, technical, or 
professional assistance, (p. 29) 

As STEP teachers had the opportunity to truly immerse themselves in the co- 
development and implementation of standards-based reform, they reminded us that the price 
of exclusion is costly. Their professional expertise and that of other PASS teachers have 
contributed to the continuous improvement of PASS framework of standards, assessments, 
criteria, scoring guides and professional development activities. Their voices can be read in 
many of our publications as successive iterations of standards, scoring guides and assessment 
guidelines. We have welcomed the prior experiences, knowledge, and personal philosophies 
that they have brought to the co-development effort. We have encouraged them to share what 
they deemed relevant to inform our discussions and development. 

The major findings of this study have also informed the next steps in the development of 
standards-based reform in Oregon. As teachers have strongly urged, the framework of standards 
and assessments for admissions to higher education (PASS) have been refined and aligned with 
the standards for the K-12 Certificates of Initial and Advanced Mastery (CIM and CAM). More 
challenging than standards' alignment is the task of aligning assessments including criteria and 
differing scoring guides. As STEP teachers indicated the alignment of these tools for assessment 
is critical in order for teachers to reach comparable judgements. To that end, current PASS 
development efforts systematically solicit teachers’ input on how CIM work samples might 
contribute to PASS collections. Teachers are also providing the raw material, including 
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classroom tasks and comments on proficiency and sufficiency criteria for assessment guidelines 
that will be used by teachers all over the state in assembling PASS collections. 

Teacher preparation programs and the requirements for continuing professional 
development have been somewhat influenced by STEP teachers' experiences with the 
verification process. In various work groups of teacher education faculty, STEP findings have 
been shared and STEP teachers have in some cases facilitated discussion groups on the 
experience of teaching in a standards-based system. Training materials that have been developed 
in the process of working with STEP teachers are being modified for use in pre-service 
coursework and practica. STEP teachers have also presented to classes of pre-service teachers in 
some of the preparation programs around the state. Performance assessment for initial and 
continuing licensure has been adopted for statewide implementation in all teacher education 
institutions as part of the policy requiring program redesign. This performance assessment will 
be revised to include evidence of a teacher’s ability to teach to a standard, generate student work 
samples, and make judgments about the level of student progress towards the standard. 

The Oregon University System through PASS is currently engaged in a collaborative 
planning effort with the Oregon Department of Education to determine the professional 
development framework that is needed to support standard-based teaching and learning. This 
professional development framework includes training and support materials generated from the 
work of STEP teachers as they participated in the cycle of activities during the second year of the 
study. This includes the framework for targeting specific standards for teaching and learning, 
planning instruction and assessment around that target, teaching with a variety of strategies to 
ensure student progress toward that standard, assessing student learning across time with 
multiple measures, verifying their judgements of student proficiency with other team members 
through the cross-scoring of student work and reflecting upon their experiences. Teachers who 
participated in STEP and PASS are frequently called upon to serve in leadership roles on district 
and state level committees that are being charged with the further development and refinement of 
professional development frameworks as well as standards and assessments. 

Finally, the voices of teachers that describe the challenges and possibilities of 
standards implementation are gaining the attention of educational leaders throughout the 
system. The implementation successes and failures of teachers provide a reference for future 
planning and development efforts across participating agencies. As Oregon struggles with the 
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challenges of implementing standards across the K-16 system, we would be wise to keep 
avenues open in the policy, teacher preparation and professional development arenas for the 
active participation of teachers who will ultimately serve as agents of successful 
implementation. 
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